Describing Multimodal Human-Computer Interaction
نویسندگان
چکیده
In this paper we argue that approaches to annotate multimodal human face-to-face interaction are not suitable for current device-based human-computer interaction. Instead, existing extensions proposed to established parameters describing the interaction with spoken dialog systems are presented and discussed. A standardization activity for annotating user interactions with multimodal systems is needed, e.g. to efficiently extract multimodal interaction parameters useful for evaluation.
منابع مشابه
Model-based Measurement of Human-Computer Interaction in Mobile Multimodal Environments
The complexity in developing and evaluating user interfaces has been extremely increased in the last few years, because more and more devices offer capabilities for multimodal interaction. This applies in particular to mobile devices like smartphones and tablet computers. An existing parameter set, aimed at describing aspects of various modalities, was extended and modified to obtain a formal, ...
متن کاملSemantic Based Information Fusion in a Multimodal Interface
The fusion of different input channels is a key factor for multimodal human-computer interfaces. In this work, we present a new concept for modality fusion. Our aim was an interface architecture that is restricted neither to a specific application field nor to specific input commands. We use a semantic network in combination with a dictionary for describing the possible user inputs – the intera...
متن کاملSoftware design and development of multimodal interaction
The multimodal dimension of a user interface raises numerous problems that are not present in more traditional interfaces. In this paper, we briefly review the current approaches in software design and modality integration techniques for multimodal interaction. We then propose a simple framework for describing multimodal interaction designs and for combining sets of user inputs of different mod...
متن کاملVocabularies for Description of Accessibility Issues in Multimodal User Interfaces
In previous work, we proposed a unified approach for describing multimodal human-computer interaction and interaction constraints in terms of sensual, motor, perceptual and cognitive functions of users. In this paper, we extend this work by providing formalised vocabularies that express human functionalities and anatomical structures required by specific modalities. The central theme of our app...
متن کاملMultimodal Human Computer Interaction: A Survey
In this paper, we review the major approaches to multimodal human–computer interaction, giving an overview of the field from a computer vision perspective. In particular, we focus on body, gesture, gaze, and affective interaction (facial expression recognition and emotion in audio). We discuss user and task modeling, and multimodal fusion, highlighting challenges, open issues, and emerging appl...
متن کامل